CONDITIONAL PROBABILITY

Learning Objectives

1. To learn the concept of a conditional probability and how to

compute it.
2. To learn the concept of independence of events, and how to
apply it.

Source: Lectures on Probability and Statistics by Dr. Eusebius Doedel

Giving more information can change the probability of an event.
EXAMPLE :
If a coin is tossed two times then what is the probability of two
Heads?
1
ANSWER : .
4

EXAMPLE :
If a coin is tossed two times then what is the probability of two Heads,
given that the first toss gave Heads ?
ANSWER : 1
.
2

1
NOTE :

Several examples will be about playing cards .

A standard deck of playing cards consists of 52 cards :

• Four suits :
Hearts , Diamonds (red ) , and Spades , Clubs (black) .

• Each suit has 13 cards, whose denomination is

2 , 3 , · · · , 10 , Jack , Queen , King , Ace .

• The Jack , Queen , and King are called face cards .

2
EXERCISE :

Suppose we draw a card from a shuffled set of 52 playing cards.

• What is the probability of drawing a Queen ?

• What is the probability of drawing a Queen, given that the card

drawn is of suit Hearts ?

• What is the probability of drawing a Queen, given that the card

drawn is a Face card ?

What do the answers tell us?

(We’ll soon learn the events ”Queen” and ”Hearts” are independent .)

3
The two preceding questions are examples of conditional probability .

Conditional probability is an important and useful concept.

If E and F are events, i.e., subsets of a sample space S , then

P (E|F ) is the conditional probability of E , given F ,

defined as
P (EF )
P (E|F ) ≡ .
P (F )
or, equivalently

P (EF ) = P (E|F ) P (F ) ,

(assuming that P (F ) is not zero).

4
 P (EF )
P (E|F ) ≡
P (F )

E
S S E
F
F

Suppose that the 6 outcomes in S are equally likely.

What is P (E|F ) in each of these two cases ?

5
 P (EF )
P (E|F ) ≡
P (F )

S E S F

Suppose that the 6 outcomes in S are equally likely.

What is P (E|F ) in each of these two cases ?

6
EXAMPLE : Suppose a coin is tossed two times.

The sample space is

Let E be the event ”two Heads ” , i.e.,

E = {HH} .

Let F be the event ”the first toss gives Heads ” , i.e.,

F = {HH , HT } .
Then
EF = {HH} = E ( since E ⊂ F ) .

We have
P (EF ) P (E) 1
1
P (E|F ) = = = 4
= .
P (F ) P (F ) 2
4
2

7
EXAMPLE :
Suppose we draw a card from a shuffled set of 52 playing cards.
• What is the probability of drawing a Queen, given that the card
drawn is of suit Hearts ?
ANSWER :
P (QH) 1
1
P (Q|H) = = 52
= .
P (H) 13
52
13

• What is the probability of drawing a Queen, given that the card

drawn is a Face card ?
ANSWER :
P (QF ) P (Q) 4
1
P (Q|F ) = = = 52
= .
P (F ) P (F ) 12
52
3

(Here Q ⊂ F , so that QF = Q .)

8
The probability of an event E is sometimes computed more easily

if we condition E on another event F ,

namely, from

P (E) = P ( E(F ∪ F c ) ) ( Why ? )

= P ( EF ∪ EF c ) = P (EF ) + P (EF c ) ( Why ? )

and

P (EF ) = P (E|F ) P (F ) , P (EF c ) = P (E|F c ) P (F c ) ,

we obtain this basic formula

P (E) = P (E|F ) · P (F ) + P (E|F c ) · P (F c ) .

9
EXAMPLE :

An insurance company has these data :

The probability of an insurance claim in a period of one year is

4 percent for persons under age 30

2 percent for persons over age 30

and it is known that

30 percent of the targeted population is under age 30.

What is the probability of an insurance claim in a period of one year

for a randomly chosen person from the targeted population?

10
SOLUTION :

Let the sample space S be all persons under consideration.

Let C be the event (subset of S) of persons filing a claim.

Let U be the event (subset of S) of persons under age 30.

Then U c is the event (subset of S) of persons over age 30.

Thus
P (C) = P (C|U ) P (U ) + P (C|U c ) P (U c )
4 3 2 7
= +
100 10 100 10
26
= = 2.6% .
1000

11
EXAMPLE :

Two balls are drawn from a bag with 2 white and 3 black balls.

There are 20 outcomes (sequences) in S . ( Why ? )

What is the probability that the second ball is white ?

SOLUTION :

Let F be the event that the first ball is white.

Let S be the event that the second second ball is white.

Then
1 2 2 3 2
P (S) = P (S|F ) P (F ) + P (S|F c ) P (F c ) = · + · = .
4 5 4 5 5

QUESTION : Is it surprising that P (S) = P (F ) ?

12
EXAMPLE : ( continued · · · )
Is it surprising that P (S) = P (F ) ?

ANSWER : Not really, if one considers the sample space S :


w 1 w2 , w1 b 1 , w1 b 2 , w1 b 3 ,

w2 w1 , w2 b 1 , w2 b 2 , w2 b 3 ,

b 1 w1 , b 1 w2 , b1 b2 , b1 b3 ,

b 2 w1 , b 2 w2 , b2 b1 , b2 b3 ,

b 3 w1 , b 3 w2 , b3 b1 , b3 b2 ,

where outcomes (sequences) are assumed equally likely.

13
EXAMPLE :

Suppose we draw two cards from a shuffled set of 52 playing cards.

What is the probability that the second card is a Queen ?

ANSWER :

P (2nd card Q) =

P (2nd card Q|1st card Q) · P (1st card Q)

+ P (2nd card Q|1st card not Q) · P (1st card not Q)

3 4 4 48 204 4 1
= · + · = = = .
51 52 51 52 51 · 52 52 13

QUESTION : Is it surprising that P (2nd card Q) = P (1st card Q) ?

14
A useful formula that ”inverts conditioning ” is derived as follows :

Since we have both

P (EF ) = P (E|F ) P (F ) ,

and
P (EF ) = P (F |E) P (E) .

If P (E) = 0 then it follows that

P (EF ) P (E|F ) · P (F )
P (F |E) = = ,
P (E) P (E)
and, using the earlier useful formula, we get
P (E|F ) · P (F )
P (F |E) = ,
P (E|F ) · P (F ) + P (E|F c ) · P (F c )

which is known as Bayes’ formula .

15
EXAMPLE : Suppose 1 in 1000 persons has a certain disease.
A test detects the disease in 99 % of diseased persons.
The test also ”detects” the disease in 5 % of healthly persons.
With what probability does a positive test diagnose the disease?

SOLUTION : Let
D ∼ ”diseased” , H ∼ ”healthy” , + ∼ ”positive”.
We are given that
P (D) = 0.001 , P (+|D) = 0.99 , P (+|H) = 0.05 .

By Bayes’ formula
P (+|D) · P (D)
P (D|+) =
P (+|D) · P (D) + P (+|H) · P (H)

0.99 · 0.001 ∼
= = 0.0194 (!)
0.99 · 0.001 + 0.05 · 0.999

16
EXERCISE :
Suppose 1 in 100 products has a certain defect.

A test detects the defect in 95 % of defective products.

The test also ”detects” the defect in 10 % of non-defective products.

• With what probability does a positive test diagnose a defect?

EXERCISE :
Suppose 1 in 2000 persons has a certain disease.

A test detects the disease in 90 % of diseased persons.

The test also ”detects” the disease in 5 % of healthly persons.

• With what probability does a positive test diagnose the disease?

17
More generally, if the sample space S is the union of disjoint events
S = F1 ∪ F2 ∪ · · · ∪ Fn ,
then for any event E
P (E|Fi ) · P (Fi )
P (Fi |E) =
P (E|F1 ) · P (F1 ) + P (E|F2 ) · P (F2 ) + · · · + P (E|Fn ) · P (Fn )

EXERCISE :
Machines M1 , M2 , M3 produce these proportions of a article

Production : M1 : 10 % , M2 : 30 % , M3 : 60 % .

The probability the machines produce defective articles is

Defects : M1 : 4 % , M2 : 3 % , M3 : 2 % .

What is the probability a random article was made by machine M1 ,

given that it is defective?

18
Independent Events

Two events E and F are independent if

P (EF ) = P (E) P (F ) .

In this case
P (EF ) P (E) P (F )
P (E|F ) = = = P (E) ,
P (F ) P (F )

(assuming P (F ) is not zero).

Thus

knowing F occurred doesn’t change the probability of E .

19
EXAMPLE : Draw one card from a deck of 52 playing cards.
Counting outcomes we find

P (Face Card) = 12
52
= 3
13
,

P (Hearts) = 13
52
= 1
4
,

P (Face Card and Hearts) = 3

52
,

P (Face Card|Hearts) = 3
13
.
We see that
3
P (Face Card and Hearts) = P (Face Card) · P (Hearts) (= ).
52
Thus the events ”Face Card ” and ”Hearts ” are independent.

Therefore we also have

3
P (Face Card|Hearts) = P (Face Card) (= ).
13
20
EXERCISE :

Which of the following pairs of events are independent?

(1) drawing ”Hearts” and drawing ”Black” ,

(2) drawing ”Black” and drawing ”Ace” ,

(3) the event {2, 3, · · · , 9} and drawing ”Red” .

21
EXERCISE : Two numbers are drawn at random from the set
{1, 2, 3, 4}.

If order is not important then what is the sample space S ?

Define the following functions on S :

X( {i, j} ) = i + j , Y ( {i, j} ) = |i − j| .

Which of the following pairs of events are independent?

(1) X = 5 and Y = 2 ,

(2) X = 5 and Y = 1 .

REMARK :
X and Y are examples of random variables . (More soon!)

22
EXAMPLE : If E and F are independent then so are E and F c .

PROOF : E = E(F ∪ F c ) = EF ∪ EF c , where

EF and EF c are disjoint .
Thus
P (E) = P (EF ) + P (EF c ) ,
from which

P (EF c ) = P (E) − P (EF )

= P (E) − P (E) · P (F ) (since E and F independent)

= P (E) · ( 1 − P (F ) )

= P (E) · P (F c ) .

EXERCISE :
Prove that if E and F are independent then so are E c and F c .

23
NOTE : Independence and disjointness are different things !

E
S S E
F
F

Independent, but not disjoint. Disjoint, but not independent.

(The six outcomes in S are assumed to have equal probability.)

If E and F are independent then P (EF ) = P (E) P (F ) .

If E and F are disjoint then P (EF ) = P ( ∅ ) = 0 .

If E and F are independent and disjoint then one has zero probability !

24
Three events E , F , and G are independent if

P (EF G) = P (E) P (F ) P (G) .

and
P (EF ) = P (E) P (F ) .
P (EG) = P (E) P (G) .
P (F G) = P (F ) P (G) .

EXERCISE : Are the three events of drawing

(1) a red card ,

(2) a face card ,

(3) a Heart or Spade ,

independent ?

25
EXERCISE :

A machine M consists of three independent parts, M1 , M2 , and M3 .

Suppose that
M1 functions properly with probability 9
10
,

M2 functions properly with probability 9

10
,

M3 functions properly with probability 8

10
,
and that

the machine M functions if and only if its three parts function.

• What is the probability for the machine M to function ?

• What is the probability for the machine M to malfunction ?

26